-
Notifications
You must be signed in to change notification settings - Fork 24.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(ngcc): read dependencies from entry-point manifest #36486
perf(ngcc): read dependencies from entry-point manifest #36486
Conversation
b8f1f8a
to
2734b3e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
for (const basePath of this.basePaths) { | ||
const entryPoints = this.entryPointManifest.readEntryPointsUsingManifest(basePath) || | ||
this.walkBasePathForPackages(basePath); | ||
unsortedEntryPoints.push(...entryPoints); | ||
entryPoints.forEach(e => unsortedEntryPoints.push(e)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this change really needed? I prefer the original code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I was looking at the generated JS code for this library, I noticed that the current code gets downleveled to (line 43):
unsortedEntryPoints.push.apply(unsortedEntryPoints, __spread(entryPoints));
See this playground
where __spread()
is implemented like:
var __read = (this && this.__read) || function (o, n) {
var m = typeof Symbol === "function" && o[Symbol.iterator];
if (!m) return o;
var i = m.call(o), r, ar = [], e;
try {
while ((n === void 0 || n-- > 0) && !(r = i.next()).done) ar.push(r.value);
}
catch (error) { e = { error: error }; }
finally {
try {
if (r && !r.done && (m = i["return"])) m.call(i);
}
finally { if (e) throw e.error; }
}
return ar;
};
var __spread = (this && this.__spread) || function () {
for (var ar = [], i = 0; i < arguments.length; i++) ar = ar.concat(__read(arguments[i]));
return ar;
};
This seemed to me highly sub-optimal and the approach in this PR is much simpler - most likely faster and certainly creates less memory pressure.
This brings up a side-point. If ngcc (and ngc) is always going to be run with node.js >= 10.0 then perhaps we should change our downleveling settings?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that's a fair point. __spread
usually shows up in profiling sessions quite noticeable so I would love to get rid of it. Emitting to ES2015 seems like the best way to achieve that, indeed.
2734b3e
to
7fd01da
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be great to see some numbers. Other than that looks great ✨
Previously, even if an entry-point did not need to be processed, ngcc would always parse the files of the entry-point to compute its dependencies. This can take a lot of time for large node_modules. Now these dependencies are cached in the entry-point manifest, and read from there rather than computing them every time. See angular#36414 FW-2047
7fd01da
to
ddb50af
Compare
Initial tests on AIO... I first built all the packages in the
So it can be seen that there is no real difference if the manifest has to be created. |
The base path for package and entry-points is known so there is no need to store these in the file. Also this commit avoids storing empty arrays unnecessarily.
a320438
to
93d7b5d
Compare
Further improvements, particularly only loading the
|
Array<AbsoluteFsPath>?, | ||
Array<AbsoluteFsPath|PathSegment>?, | ||
Array<AbsoluteFsPath>?, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it make sense to use relative paths for these too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about it for dependencies
and deep-imports
even though there is a chance that they might start with ..
(but that probably doesn't matter). But I didn't want to do it for missing
since some of these are PathSegment
s but relative to the import place in the code, so we couldn't be sure, when reading in the manifest, whether we are reading in an AbsolutePath that needs de-relativising or a PathSegment that needs to be left alone.
@@ -315,6 +314,7 @@ export function mainNgcc({ | |||
const createCompileFn: CreateCompileFn = onTaskCompleted => { | |||
const fileWriter = getFileWriter( | |||
fileSystem, logger, pkgJsonUpdater, createNewEntryPointFormats, errorOnFailedEntryPoint); | |||
const {Transformer} = require('./packages/transformer'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume we can do more of this. Is Trasnformer
special in some way (e.g. particularly heavy to load)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, Transformer brings in all the trait compiler stuff, all the reflection hosts etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked at other imports but most of them are needed in the top level main function :-/
Previously, even if an entry-point did not need to be processed, ngcc would always parse the files of the entry-point to compute its dependencies. This can take a lot of time for large node_modules. Now these dependencies are cached in the entry-point manifest, and read from there rather than computing them every time. See #36414 FW-2047 PR Close #36486
The base path for package and entry-points is known so there is no need to store these in the file. Also this commit avoids storing empty arrays unnecessarily. PR Close #36486
The base path for package and entry-points is known so there is no need to store these in the file. Also this commit avoids storing empty arrays unnecessarily. PR Close #36486
This issue has been automatically locked due to inactivity. Read more about our automatic conversation locking policy. This action has been performed automatically by a bot. |
Previously, even if an entry-point did not need to be processed,
ngcc would always parse the files of the entry-point to compute
its dependencies. This can take a lot of time for large node_modules.
Now these dependencies are cached in the entry-point manifest,
and read from there rather than computing them every time.
See #36414 (comment)
FW-2047